

<!DOCTYPE html>
<html class="writer-html5" lang="en" >
<head>
  <meta charset="utf-8" />
  
  <meta name="viewport" content="width=device-width, initial-scale=1.0" />
  
  <title>Orphan List and Associated Tooling &mdash; Ceph Documentation</title>
  

  
  <link rel="stylesheet" href="../../_static/ceph.css" type="text/css" />
  <link rel="stylesheet" href="../../_static/pygments.css" type="text/css" />
  <link rel="stylesheet" href="../../_static/graphviz.css" type="text/css" />
  <link rel="stylesheet" href="../../_static/css/custom.css" type="text/css" />

  
  
    <link rel="shortcut icon" href="../../_static/favicon.ico"/>
  

  
  

  

  
  <!--[if lt IE 9]>
    <script src="../../_static/js/html5shiv.min.js"></script>
  <![endif]-->
  
    
      <script type="text/javascript" id="documentation_options" data-url_root="../../" src="../../_static/documentation_options.js"></script>
        <script src="../../_static/jquery.js"></script>
        <script src="../../_static/underscore.js"></script>
        <script src="../../_static/doctools.js"></script>
    
    <script type="text/javascript" src="../../_static/js/theme.js"></script>

    
    <link rel="index" title="Index" href="../../genindex/" />
    <link rel="search" title="Search" href="../../search/" />
    <link rel="next" title="OpenID Connect Provider in RGW" href="../oidc/" />
    <link rel="prev" title="Session tags for Attribute Based Access Control in STS" href="../session-tags/" /> 
</head>

<body class="wy-body-for-nav">

   
  <header class="top-bar">
    

















<div role="navigation" aria-label="breadcrumbs navigation">

  <ul class="wy-breadcrumbs">
    
      <li><a href="../../" class="icon icon-home"></a> &raquo;</li>
        
          <li><a href="../">Ceph 对象网关</a> &raquo;</li>
        
      <li>Orphan List and Associated Tooling</li>
    
    
      <li class="wy-breadcrumbs-aside">
        
          
            <a href="../../_sources/radosgw/orphans.rst.txt" rel="nofollow"> View page source</a>
          
        
      </li>
    
  </ul>

  
  <hr/>
</div>
  </header>
  <div class="wy-grid-for-nav">
    
    <nav data-toggle="wy-nav-shift" class="wy-nav-side">
      <div class="wy-side-scroll">
        <div class="wy-side-nav-search"  style="background: #eee" >
          

          
            <a href="../../">
          

          
            
            <img src="../../_static/logo.png" class="logo" alt="Logo"/>
          
          </a>

          

          
<div role="search">
  <form id="rtd-search-form" class="wy-form" action="../../search/" method="get">
    <input type="text" name="q" placeholder="Search docs" />
    <input type="hidden" name="check_keywords" value="yes" />
    <input type="hidden" name="area" value="default" />
  </form>
</div>

          
        </div>

        
        <div class="wy-menu wy-menu-vertical" data-spy="affix" role="navigation" aria-label="main navigation">
          
            
            
              
            
            
              <ul class="current">
<li class="toctree-l1"><a class="reference internal" href="../../start/intro/">Ceph 简介</a></li>
<li class="toctree-l1"><a class="reference internal" href="../../install/">安装 Ceph</a></li>
<li class="toctree-l1"><a class="reference internal" href="../../cephadm/">Cephadm</a></li>
<li class="toctree-l1"><a class="reference internal" href="../../rados/">Ceph 存储集群</a></li>
<li class="toctree-l1"><a class="reference internal" href="../../cephfs/">Ceph 文件系统</a></li>
<li class="toctree-l1"><a class="reference internal" href="../../rbd/">Ceph 块设备</a></li>
<li class="toctree-l1 current"><a class="reference internal" href="../">Ceph 对象网关</a><ul class="current">
<li class="toctree-l2"><a class="reference internal" href="../frontends/">HTTP 前端</a></li>
<li class="toctree-l2"><a class="reference internal" href="../placement/">存储池归置与存储类</a></li>
<li class="toctree-l2"><a class="reference internal" href="../multisite/">多站配置</a></li>
<li class="toctree-l2"><a class="reference internal" href="../multisite-sync-policy/">多站同步策略配置</a></li>
<li class="toctree-l2"><a class="reference internal" href="../pools/">存储池的配置</a></li>
<li class="toctree-l2"><a class="reference internal" href="../config-ref/">配置参考</a></li>
<li class="toctree-l2"><a class="reference internal" href="../admin/">管理指南</a></li>
<li class="toctree-l2"><a class="reference internal" href="../s3/">S3 API</a></li>
<li class="toctree-l2"><a class="reference internal" href="../rgw-cache/">Data caching and CDN</a></li>
<li class="toctree-l2"><a class="reference internal" href="../swift/">Swift API</a></li>
<li class="toctree-l2"><a class="reference internal" href="../adminops/">管理操作 API</a></li>
<li class="toctree-l2"><a class="reference internal" href="../api/">Python 接口</a></li>
<li class="toctree-l2"><a class="reference internal" href="../nfs/">通过 NFS 导出</a></li>
<li class="toctree-l2"><a class="reference internal" href="../keystone/">与 OpenStack Keystone 对接</a></li>
<li class="toctree-l2"><a class="reference internal" href="../barbican/">与 OpenStack Barbican 对接</a></li>
<li class="toctree-l2"><a class="reference internal" href="../vault/">与 HashiCorp Vault 对接</a></li>
<li class="toctree-l2"><a class="reference internal" href="../kmip/">KMIP Integration</a></li>
<li class="toctree-l2"><a class="reference internal" href="../opa/">与 Open Policy Agent 对接</a></li>
<li class="toctree-l2"><a class="reference internal" href="../multitenancy/">多租户</a></li>
<li class="toctree-l2"><a class="reference internal" href="../compression/">压缩</a></li>
<li class="toctree-l2"><a class="reference internal" href="../ldap-auth/">LDAP 认证</a></li>
<li class="toctree-l2"><a class="reference internal" href="../encryption/">服务器端加密</a></li>
<li class="toctree-l2"><a class="reference internal" href="../bucketpolicy/">桶策略</a></li>
<li class="toctree-l2"><a class="reference internal" href="../dynamicresharding/">动态的桶索引重分片</a></li>
<li class="toctree-l2"><a class="reference internal" href="../mfa/">多因子认证</a></li>
<li class="toctree-l2"><a class="reference internal" href="../sync-modules/">同步模块</a></li>
<li class="toctree-l2"><a class="reference internal" href="../notifications/">Bucket Notifications</a></li>
<li class="toctree-l2"><a class="reference internal" href="../layout/">RADOS 中的数据布局</a></li>
<li class="toctree-l2"><a class="reference internal" href="../STS/">STS</a></li>
<li class="toctree-l2"><a class="reference internal" href="../STSLite/">STS Lite</a></li>
<li class="toctree-l2"><a class="reference internal" href="../keycloak/">Keycloak</a></li>
<li class="toctree-l2"><a class="reference internal" href="../role/">Role</a></li>
<li class="toctree-l2"><a class="reference internal" href="../session-tags/">Session Tags</a></li>
<li class="toctree-l2 current"><a class="current reference internal" href="#">Orphan List and Associated Tooliing</a><ul>
<li class="toctree-l3"><a class="reference internal" href="#orphans-find-deprecated">Orphans Find – DEPRECATED</a></li>
<li class="toctree-l3"><a class="reference internal" href="#orphan-list">Orphan List</a><ul>
<li class="toctree-l4"><a class="reference internal" href="#warning-experimental-status">WARNING: Experimental Status</a></li>
<li class="toctree-l4"><a class="reference internal" href="#warning-specifying-a-data-pool">WARNING: Specifying a Data Pool</a></li>
<li class="toctree-l4"><a class="reference internal" href="#warning-unindexed-buckets">WARNING: Unindexed Buckets</a></li>
</ul>
</li>
<li class="toctree-l3"><a class="reference internal" href="#rados-list">RADOS List</a><ul>
<li class="toctree-l4"><a class="reference internal" href="#note-shared-bucket-markers">Note: Shared Bucket Markers</a></li>
</ul>
</li>
</ul>
</li>
<li class="toctree-l2"><a class="reference internal" href="../oidc/">OpenID Connect Provider</a></li>
<li class="toctree-l2"><a class="reference internal" href="../troubleshooting/">故障排除</a></li>
<li class="toctree-l2"><a class="reference internal" href="../../man/8/radosgw/">radosgw 手册页</a></li>
<li class="toctree-l2"><a class="reference internal" href="../../man/8/radosgw-admin/">radosgw-admin 手册页</a></li>
<li class="toctree-l2"><a class="reference internal" href="../qat-accel/">使用 QAT 为加密和压缩提速</a></li>
<li class="toctree-l2"><a class="reference internal" href="../s3select/">S3-select</a></li>
<li class="toctree-l2"><a class="reference internal" href="../lua-scripting/">Lua Scripting</a></li>
<li class="toctree-l2"><a class="reference internal" href="../d3n_datacache/">D3N Data Cache</a></li>
<li class="toctree-l2"><a class="reference internal" href="../cloud-transition/">Cloud Transition</a></li>
</ul>
</li>
<li class="toctree-l1"><a class="reference internal" href="../../mgr/">Ceph 管理器守护进程</a></li>
<li class="toctree-l1"><a class="reference internal" href="../../mgr/dashboard/">Ceph 仪表盘</a></li>
<li class="toctree-l1"><a class="reference internal" href="../../api/">API 文档</a></li>
<li class="toctree-l1"><a class="reference internal" href="../../architecture/">体系结构</a></li>
<li class="toctree-l1"><a class="reference internal" href="../../dev/developer_guide/">开发者指南</a></li>
<li class="toctree-l1"><a class="reference internal" href="../../dev/internals/">Ceph 内幕</a></li>
<li class="toctree-l1"><a class="reference internal" href="../../governance/">项目管理</a></li>
<li class="toctree-l1"><a class="reference internal" href="../../foundation/">Ceph 基金会</a></li>
<li class="toctree-l1"><a class="reference internal" href="../../ceph-volume/">ceph-volume</a></li>
<li class="toctree-l1"><a class="reference internal" href="../../releases/general/">Ceph 版本（总目录）</a></li>
<li class="toctree-l1"><a class="reference internal" href="../../releases/">Ceph 版本（索引）</a></li>
<li class="toctree-l1"><a class="reference internal" href="../../security/">Security</a></li>
<li class="toctree-l1"><a class="reference internal" href="../../glossary/">Ceph 术语</a></li>
<li class="toctree-l1"><a class="reference internal" href="../../jaegertracing/">Tracing</a></li>
<li class="toctree-l1"><a class="reference internal" href="../../translation_cn/">中文版翻译资源</a></li>
</ul>

            
          
        </div>
        
      </div>
    </nav>

    <section data-toggle="wy-nav-shift" class="wy-nav-content-wrap">

      
      <nav class="wy-nav-top" aria-label="top navigation">
        
          <i data-toggle="wy-nav-top" class="fa fa-bars"></i>
          <a href="../../">Ceph</a>
        
      </nav>


      <div class="wy-nav-content">
        
        <div class="rst-content">
        
          <div role="main" class="document" itemscope="itemscope" itemtype="http://schema.org/Article">
           <div itemprop="articleBody">
            
<div id="dev-warning" class="admonition note">
  <p class="first admonition-title">Notice</p>
  <p class="last">This document is for a development version of Ceph.</p>
</div>
  <div id="docubetter" align="right" style="padding: 5px; font-weight: bold;">
    <a href="https://pad.ceph.com/p/Report_Documentation_Bugs">Report a Documentation Bug</a>
  </div>

  
  <div class="section" id="orphan-list-and-associated-tooling">
<h1><a class="toc-backref" href="#id1">Orphan List and Associated Tooling</a><a class="headerlink" href="#orphan-list-and-associated-tooling" title="Permalink to this headline">¶</a></h1>
<div class="contents topic" id="contents">
<p class="topic-title">Contents</p>
<ul class="simple">
<li><p><a class="reference internal" href="#orphan-list-and-associated-tooling" id="id1">Orphan List and Associated Tooling</a></p>
<ul>
<li><p><a class="reference internal" href="#orphans-find-deprecated" id="id2">Orphans Find – DEPRECATED</a></p></li>
<li><p><a class="reference internal" href="#orphan-list" id="id3">Orphan List</a></p>
<ul>
<li><p><a class="reference internal" href="#warning-experimental-status" id="id4">WARNING: Experimental Status</a></p></li>
<li><p><a class="reference internal" href="#warning-specifying-a-data-pool" id="id5">WARNING: Specifying a Data Pool</a></p></li>
<li><p><a class="reference internal" href="#warning-unindexed-buckets" id="id6">WARNING: Unindexed Buckets</a></p></li>
</ul>
</li>
<li><p><a class="reference internal" href="#rados-list" id="id7">RADOS List</a></p>
<ul>
<li><p><a class="reference internal" href="#note-shared-bucket-markers" id="id8">Note: Shared Bucket Markers</a></p></li>
</ul>
</li>
</ul>
</li>
</ul>
</div>
<p>Orphans are RADOS objects that are left behind after their associated
RGW objects are removed. Normally these RADOS objects are removed
automatically, either immediately or through a process known as
“garbage collection”. Over the history of RGW, however, there may have
been bugs that prevented these RADOS objects from being deleted, and
these RADOS objects may be consuming space on the Ceph cluster without
being of any use. From the perspective of RGW, we call such RADOS
objects “orphans”.</p>
<div class="section" id="orphans-find-deprecated">
<h2><a class="toc-backref" href="#id2">Orphans Find – DEPRECATED</a><a class="headerlink" href="#orphans-find-deprecated" title="Permalink to this headline">¶</a></h2>
<p>The <cite>radosgw-admin</cite> tool has/had three subcommands to help manage
orphans, however these subcommands are (or will soon be)
deprecated. These subcommands are:</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><style type="text/css">
span.prompt1:before {
  content: "# ";
}
</style><span class="prompt1">radosgw-admin orphans find ...</span>
<span class="prompt1">radosgw-admin orphans finish ...</span>
<span class="prompt1">radosgw-admin orphans list-jobs ...</span>
</pre></div></div><p>There are two key problems with these subcommands, however. First,
these subcommands have not been actively maintained and therefore have
not tracked RGW as it has evolved in terms of features and updates. As
a result the confidence that these subcommands can accurately identify
true orphans is presently low.</p>
<p>Second, these subcommands store intermediate results on the cluster
itself. This can be problematic when cluster administrators are
confronting insufficient storage space and want to remove orphans as a
means of addressing the issue. The intermediate results could strain
the existing cluster storage capacity even further.</p>
<p>For these reasons “orphans find” has been deprecated.</p>
</div>
<div class="section" id="orphan-list">
<h2><a class="toc-backref" href="#id3">Orphan List</a><a class="headerlink" href="#orphan-list" title="Permalink to this headline">¶</a></h2>
<p>Because “orphans find” has been deprecated, RGW now includes an
additional tool – ‘rgw-orphan-list’. When run it will list the
available pools and prompt the user to enter the name of the data
pool. At that point the tool will, perhaps after an extended period of
time, produce a local file containing the RADOS objects from the
designated pool that appear to be orphans. The administrator is free
to examine this file and the decide on a course of action, perhaps
removing those RADOS objects from the designated pool.</p>
<p>All intermediate results are stored on the local file system rather
than the Ceph cluster. So running the ‘rgw-orphan-list’ tool should
have no appreciable impact on the amount of cluster storage consumed.</p>
<div class="section" id="warning-experimental-status">
<h3><a class="toc-backref" href="#id4">WARNING: Experimental Status</a><a class="headerlink" href="#warning-experimental-status" title="Permalink to this headline">¶</a></h3>
<p>The ‘rgw-orphan-list’ tool is new and therefore currently considered
experimental. The list of orphans produced should be “sanity checked”
before being used for a large delete operation.</p>
</div>
<div class="section" id="warning-specifying-a-data-pool">
<h3><a class="toc-backref" href="#id5">WARNING: Specifying a Data Pool</a><a class="headerlink" href="#warning-specifying-a-data-pool" title="Permalink to this headline">¶</a></h3>
<p>If a pool other than an RGW data pool is specified, the results of the
tool will be erroneous. All RADOS objects found on such a pool will
falsely be designated as orphans.</p>
</div>
<div class="section" id="warning-unindexed-buckets">
<h3><a class="toc-backref" href="#id6">WARNING: Unindexed Buckets</a><a class="headerlink" href="#warning-unindexed-buckets" title="Permalink to this headline">¶</a></h3>
<p>RGW allows for unindexed buckets, that is buckets that do not maintain
an index of their contents. This is not a typical configuration, but
it is supported. Because the ‘rgw-orphan-list’ tool uses the bucket
indices to determine what RADOS objects should exist, objects in the
unindexed buckets will falsely be listed as orphans.</p>
</div>
</div>
<div class="section" id="rados-list">
<h2><a class="toc-backref" href="#id7">RADOS List</a><a class="headerlink" href="#rados-list" title="Permalink to this headline">¶</a></h2>
<p>One of the sub-steps in computing a list of orphans is to map each RGW
object into its corresponding set of RADOS objects. This is done using
a subcommand of ‘radosgw-admin’.</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span class="prompt1">radosgw-admin bucket radoslist <span class="o">[</span>--bucket<span class="o">={</span>bucket-name<span class="o">}]</span></span>
</pre></div></div><p>The subcommand will produce a list of RADOS objects that support all
of the RGW objects. If a bucket is specified then the subcommand will
only produce a list of RADOS objects that correspond back the RGW
objects in the specified bucket.</p>
<div class="section" id="note-shared-bucket-markers">
<h3><a class="toc-backref" href="#id8">Note: Shared Bucket Markers</a><a class="headerlink" href="#note-shared-bucket-markers" title="Permalink to this headline">¶</a></h3>
<p>Some administrators will be aware of the coding schemes used to name
the RADOS objects that correspond to RGW objects, which include a
“marker” unique to a given bucket.</p>
<p>RADOS objects that correspond with the contents of one RGW bucket,
however, may contain a marker that specifies a different bucket. This
behavior is a consequence of the “shallow copy” optimization used by
RGW. When larger objects are copied from bucket to bucket, only the
“head” objects are actually copied, and the tail objects are
shared. Those shared objects will contain the marker of the original
bucket.</p>
</div>
</div>
</div>



           </div>
           
          </div>
          <footer>
    <div class="rst-footer-buttons" role="navigation" aria-label="footer navigation">
        <a href="../oidc/" class="btn btn-neutral float-right" title="OpenID Connect Provider in RGW" accesskey="n" rel="next">Next <span class="fa fa-arrow-circle-right" aria-hidden="true"></span></a>
        <a href="../session-tags/" class="btn btn-neutral float-left" title="Session tags for Attribute Based Access Control in STS" accesskey="p" rel="prev"><span class="fa fa-arrow-circle-left" aria-hidden="true"></span> Previous</a>
    </div>

  <hr/>

  <div role="contentinfo">
    <p>
        &#169; Copyright 2016, Ceph authors and contributors. Licensed under Creative Commons Attribution Share Alike 3.0 (CC-BY-SA-3.0).

    </p>
  </div> 

</footer>
        </div>
      </div>

    </section>

  </div>
  

  <script type="text/javascript">
      jQuery(function () {
          SphinxRtdTheme.Navigation.enable(true);
      });
  </script>

  
  
    
   

</body>
</html>